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Abstract 

The aim of this study is to examine the questionnaires used to evaluate teaching performance in higher educa¬ 
tion institutes and called 'Instructor and Course Evaluation Questionnaires (ICEQ**)’ in terms of questionnaire 
preparation techniques and components of curriculum. Obtaining at least one ICEQ belonging to any state and 
private universities in Turkey is the universe of the study. The sample of the study is 35 ICEQs from 32 universi¬ 
ties. In this study, qualitative research methods are used and one of the data collection methods, document 
analysis method is applied and the documents/data are decoded by Content Analysis Method. The items of the 
questionnaires are analyzed in terms of curriculum components and questionnaire preparing techniques. The 
findings of the study indicate that the items included in the questionnaire are related mostly to the teaching- 
learning processes component of a curriculum. 52 % of all the questionnaires items are determined as related 
to benefits obtained from the courses by students and it is found out that 21% of the items are not appropriate for 
the criteria of preparing questionnaires. That the analyzed ICEQs are prepared in accordance with curriculum 
and questionnaire preparing techniques is vital because it contributes to validity and reliability of questionnaires. 
A qualified evaluation can be reached via findings obtained from reliable and valid data collection tools. It is a 
necessity to collect data through reliable and valid tools for summative and formative evaluations conducted for 
instructors and courses in higher education institutions. 
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One of important signs of quality university is 
the quality level of instructor and his/her cour¬ 
se. The quality of instructor is the leading one 
among the factors affecting directly the quality 
of education and so evaluation of instructor 
performance is considered as the assurance of 
quality (A^an & Saydan, 2009, p. 227; Karamul- 
laoglu, 2000, p. 34). 
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Due to the fact that there has been a strong de¬ 
mand for higher education, universities in Tur¬ 
key have no fear of not being able to find stu¬ 
dents to be enrolled to their programs. Because 
of this fact, a serious competition can not be seen 
among state universities of Turkey. These uni¬ 
versities only compete for talented, academically 
high performing students (Tiirker, 2003). This 
result was valid for seven years ago when it was 
stated at that time, however by 2009-2010 acade¬ 
mical year, some of the 103 state andd 53 private, 
in sum, total of 156 universities (Yiiksekogretim 

** In this study, the term ICEQ is used for "‘Instruc¬ 
tor and Course Evaluation Questionnaires” as 
a short notation. Name of each Instructor and 
Course Evaluation Questionnaire is different in 
each university where the questionnaire is used. 
For persons who are interested in this subject, 
being careful about keeping “concept unification” 
intact will increase understandability on "theory” 
and “application”. 
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Kurulu [YOK], 2010) in Turkey, are now facing 
problem of not being able to find enough stu¬ 
dents enrolled in their programs (01i;me, Seipme 
ve Yerle^tirme Merkezi [OSYM], 2010; Ozmen, 
2010). The idea of universities given by Malche 
(1932) is stil very important as it was in the past 
for our moral future and our society’s future. 
However, question of “ what has been done for 
improving these important institutions?” has to 
be asked. To answer this question adequately, 
universities need to have an effective evaluation 
system. Go^men (2004) stated that, nowadays, 
education related improvement concept conta¬ 
ins both quantitative and qualitative improve¬ 
ment at the same time. 

Without evaluation, it is hard to deal with qu¬ 
ality related problems. Moreover, as Theall and 
Franklin (2000) indicate, evaluation should not 
be regarded as a punitive process, on the cont¬ 
rary it is emphasized that evaluation should 
contribute to development. 

It is known that feedback from students and other 
sources have important functions and usages du¬ 
ring evaluating and improving curricula of hig¬ 
her education institutions. As a result of this fact, 
European and American universities give great 
importance to the feedback based on evaluation 
made by students (Moore & Kuol, 2005). In Tur¬ 
kish literature, researches on ICEQs are very limi¬ 
ted. This study is a pioneering one on the subject, 
this situation makes this study very important for 
the Turkish literature. Instructors and courses 
evaluation can be made by using various methods 
(Kalayci, 2009; Tong & Bures, 1987). 

Evaluation of teaching performance by students 
has been frequently used a method of evalua¬ 
tion since 1920 (Cashin, 1999; Seldin, 1999). 
Researchers are mostly in an agreement such 
that results of students’ evaluation teaching 
questionnaires are influential and effective in 
improving the quality of teaching (Jackson et 
al., 1999; Marsh & Roche, 1997; Perry & Smart, 
1997). However, discussions and arguments re¬ 
lated to reliability and validity of questionnaires 
still continue among researchers (Barth, Charl- 
ston, & Carolina, 2008; Cashin, 1995; Centra, 
1993; Hoyt & Palet, 1999; Kolitch & Dean, 1999; 
Marsh, 1984; Oliver, Tucker, Gupta, & Yeo, 2008; 
Theall, 2002). 

It is known that, in higher education programs, 
the feedbacks which are obtained from students 
and other sources and which are used in cur¬ 
riculum evaluation and development, have an 
important usage (Devebakan, Koijdor, Musal, & 
Giiner, 2003; K 09 & Co^kuner, 2007). As it was 


stated by Vari$ (1988), Curriculum Develop¬ 
ment is not a process of chancing some parts of 
the curriculum or is not a process of adding or 
deleting some parts of the curriculum. 

During the process of determining impor¬ 
tant points which are related to preparation of 
ICEQs, related literature has been considered 
carefully (Altum^ik, Co§kun, Bayraktaroglu, & 
Yildirim, 2004; Arikan 2007; Arseven, 2004; Bal- 
ci, 2009; Ba§, 2008; Biiyiikozturk, Qakmak, Ak- 
giin, Karadeniz, & Demirel, 2009; Cashin, 1995; 
Chen & Hashower, 2003; Ergiin, Duman, Kincal, 
& Ariba$, 1999; Fraenkel & Wallen, 2000; Hel- 
terbran, 2008; Kalayci, 2008a; Kolitch & Dean, 
1999; Marsh & Roche, 1997; Millan & Schu¬ 
macher, 2001; Ozoglu, 1992; Pamuk, 2005; San, 
2004; Tagomori & Bishop, 1995; Van?, 1998, p. 
6 ; Voss, Gruber & Szmigin, 2007) and these im¬ 
portant points were gathered under four main 
title: Items in ICEQs have to be compatible with 
curriculum, Expectations of students from edu¬ 
cation have to be known, ICEQ has to be valid 
and reliable, ICEQ has to be in accordance with 
questionnaire preparation and item wording 
principles. 

With this point of view, ICEWs which are used 
in Turkish Higher Education System have to be 
in accordance with questionnaire itself with qu¬ 
estionnaire preparation principles. There is no 
strict principles list saying how to design and 
prepare an ICEQ and also saying which type of 
items to be included in an ICEQ. Usage of ICEQs 
is advised by Higher Education Academical 
Evaluation and Quality Improvement Commis¬ 
sion (YODEK, 2007). These questionnaires are 
prepared and put into practice by Academical 
Evaluation Quality Improvement Commissi¬ 
on (AEQIC) of each university Related to this 
ICEQ preparation and application process, some 
faculties in few universities are conducting this 
process independently. During the ICEQ prepa¬ 
ration process; questionnaire preparation prin¬ 
ciples, dimensions of the curriculum, reliability 
and validity calculations have to be considered 
very careful. In general, following sections can 
be found in ICEQs: a section related to the teac¬ 
hing performance of an instructor (communica¬ 
tion, class management, evaluation etc.), a secti¬ 
on related to the evaluation of the course (course 
content, goals and objectives of the course, being 
hard course, complexity, course usefulness etc.), 
a section related to evaluation of students’ self¬ 
performance during the course. 

When the ICEQ related literature is considered 
carefully, there is no research study which anali- 
ze the ICEQs based on the dimensions of curri- 
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culum. In general, ICEQ related literature have 
focused on the variables which have effects on 
the ratings given by students (Barth et al., 2008; 
Comer, 2001, Jacobs,1995; Kalayci, 2008a; YOK, 
2000 ); evaluation methods for evaluating ins¬ 
tructors (Colins, 2002; Kalayci, 2009), teaching 
performance of instructors (Aksu, Qivitci, & 
Duy, 2008; Devebakan et al., 2003; K 09 & C 05 - 
kuner, 2007; Murat, Aslanba§, & Ozgan, 2006; 
Tonbul, 2008; Ye§ilta§ & Oztiirk, 2000) and qu¬ 
estionnaire preparation process (Oliver et al., 
2008; San, 2004). 

In this study, of all these various methods of eva¬ 
luation of teaching performance of an instructor, 
ICEQs used in Turkish universities are analyzed. 
The study shows how the questionnaires used in 
Turkish universities and their items are formed 
and how the items show distribution in terms of 
curriculum components. The findings will provide 
important clues on the process of preparing questi¬ 
onnaires; hence, the study is also important. 

Purpose 

The aim of the study is to examine the question¬ 
naires used to evaluate teaching performance at 
higher education institutions in Turkey and called 
‘Instructor and Course Evaluation Questionnai¬ 
res (ICEQ)’ in terms of questionnaire preparation 
techniques and components of curriculum. 

In order to reach the aim, the three research qu¬ 
estions below are tried to be answered. 

1. How the items which are aiming to evaluate 
teaching performance of instructors inclu¬ 
ded in ICEQ are 

formed based on curriculum components? 

2. How the items which are aiming to evaluate 
courses included in ICEQs are formed? 

3. How are ICEQs formed based on the met¬ 
hods and principles of questionnaire prepa¬ 
ration? 


Method 

Research Design 

This study aims to describe a current situati¬ 
on in the same way as it is in real life (Karasar, 
2009), so survey method is applied. Qualitati¬ 
ve research methods are applied and the study 
is designed on the basis of qualitative research 
method qualifications. Of all the qualitative data 
collection methods “written document analysis” 
is applied. 


Universe and Sampling 

The universe of the study is the whole of the ins¬ 
tructors and course evaluation questionnaires 
used in all of the state and private universities 
in Turkey. In this study, 139 universities are con¬ 
tacted including 94 state and 45 private univer¬ 
sities and in the end, 35 ICEQs from 32 Turkish 
universities constituted the sample of the study. 

Instrument 

In the light of research questions, the question¬ 
naires on students’ opinions related to teaching 
performance of instructors and courses evalu¬ 
ation in Turkish universities were collected in 
2009. 

The data of this study are the questionnaires and 
their items which are used by students to eva¬ 
luate instructors and their courses. Concerning 
this part of the study, 716 items collected from 
ICEQs are analyzed in terms of questionnaire 
preparation strategies. Of all the items, 570 of 
them are to evaluate the instructors; 100 of them 
are to evaluate the course and 46 of them are to 
evaluate the students themselves. 

Data Analysis 

Document analysis consists of analysis of writ¬ 
ten materials which includes information about 
phenomenon or phenomena to be aimed to re¬ 
search (Yildmm & $im§ek, 2000, p. 140). In this 
study data in other words questionnaires are de¬ 
coded via ‘Content Analysis Method’. 

Results 

Of all the 570 items in the questionnaires evalu¬ 
ating instructors teaching performances, 6 % are 
considered as related to the Aim component of 
a curriculum, 12 % to the Content component 
of a curriculum, 68 % to the Teaching- Learning 
Process component of a curriculum and finally 
14 % to the Assessment component of a curri¬ 
culum. With reference to the findings, it can be 
concluded that the questionnaires intending to 
evaluate instructors teaching performance focu¬ 
ses mostly on teaching- learning process compo¬ 
nent of curriculum and that other components 
of curriculum are ignored. 

According to K 09 and Co^kuner’s (2007) study 
results, the students think the questionnaire 
items are insufficient and claim that there sho¬ 
uld be more items related to the teaching tech¬ 
niques (the component of teaching learning 
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Table 1. 

The Distribution of ICEQs’ Items Evaluating Instructors Teaching Performance according to Components of Curriculum 

Curriculum 

Components 

Sub Categories 

Frequency (f) 

Percent (%) Percent (%) 

Objectives 


33 

6 

6 


Sources 

27 

5 


Content 

Qualification of instructors 

25 

4 

12 


Subject (Content) 

14 

3 



Teaching process (conveying the course) 

157 

28 


Teaching- 

Learning 

Communication 

113 

20 


Time management 

54 

9 

68 

Process 

Course preparation 

41 

7 



Teaching materials 

24 

4 


Assessment 

Measurement 

43 

8 

14 

Evaluation 

39 

6 

Total 


570 

100 

100 


process), communication skills (the component 
of teaching learning process), instructors field 
expertise (the component of content) etc. used 
in courses by instructors. Colins (2002) finds out 
that the questionnaires through which students 
evaluate teachers are inadequate in terms of con¬ 
tent and structure and advises that the question¬ 
naires be corrected and re-prepared. 

When the total 716 items are analyzed accor¬ 
ding questionnaire preparing techniques and 
strategies, 21% of the items (152 items) are not 
in line with these techniques and strategies. Of 
all the items, 10 % are not written in line with 
the criteria stated as ‘With an item, only one 
subject should be asked.’ In other words, 10% 
of the questionnaire items are written in such 
a way that these items are trying to evaluate 
more than one thing in one item. This situ¬ 
ation is completely against the questionnaire 
preparation principles. 

Some of the important studies related to wor¬ 
ding of items of questionnaires, design of the 
questionnaires, and constructing a scale are con¬ 
ducted by Low (1988, 1991, 1999) and also both 
Alderson (1992) and Low (1996) indicated that 
studies related to these variables are not adequ¬ 
ate. In our country, Turkey, universities, which 
are conducting studies, related to ICEQs, have 
to focus on two types of studies in general: the 
first one is to prepare valid and reliable ICEQs 
and the second one is to increase the number of 
studies related to ICEQs. 

The study of Colins (2002) states that in ques¬ 
tionnaires, with which students evaluates their 
instructors, there are some contradictory sta¬ 
tements ( items) and states that statements or 
wording of items have to be more open and 
more understandable. The findings of this pre¬ 


sent study are in agreement with the findings of 
the studies of Coffey and Gibbs (2001), Elusband 
(1996), and Kolitch and Dean (1999). 

Discussion 

The findings of this study show that, in Turkish 
universities, preparation of questionnaires’ items 
is not based on any systems or any techniques. 
As long as people preparing the questionnaires 
do not acquire or follow a specific system and 
techniques, the data collected through these qu¬ 
estionnaires will not be reliable and valid. This 
situation brings out ineffective evaluation and 
the questionnaires will be used only as to per¬ 
form a routine. 

In this study, when the items of ICEQs are 
analyzed in terms of questionnaire preparation 
techniques, 716 items collected from 35 questi¬ 
onnaires are analyzed and of all the items 152 of 
them are determined as not appropriate to ques¬ 
tionnaire preparation techniques. 

The finding indicates the questionnaires have 
problems regarding validity and reliability. The 
questionnaires with reliability and validity prob¬ 
lems will affect the summative and formative 
evaluation process negatively. 

At that point, Kalaycfs suggestion (2008b) rela¬ 
ted to establishing centers with the aim of incre¬ 
asing the education quality at universities is very 
important. Kalayct (2008b) claims that whatever 
its name such kind of centers should be estab¬ 
lished and especially they are vital for education 
quality. At universities with academic success 
and quality, curriculum development studies 
function as curriculum development circle at 
such centers. At the same centers, there are ex¬ 
perts varying from measurement and evaluati- 
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on experts to curriculum development experts. 
Academical Evaluation Quality Improvement 
Commission (AEQIC) may begin applications 
like the centers’ operations. 
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